In video and imaging systems, color generally is represented as vector coordinates in a multi-dimensional “color space” having three or more channels. Common examples include the well-known classes of RGB and YUV color spaces. The RGB color spaces specify pixel values using coordinates that represent intensities of red, green and blue light, respectively. The YUV color spaces specify pixel values using coordinates that represent a luminance or chrominance value.
Currently, many image capture, processing and display devices can only handle pixel values with a small dynamic range of 256 (28) discrete values per channel, represented by 8 bits. Such images can be described as having a “hit depth” of 8 bits. In a typical RGB digital image having 8 bits per channel (8 bpc) in red, green and blue color channels, only 256 different values are possible for each of the red, green and blue values. Other values (e.g., alpha or opacity values, luminance, etc.) also may be constrained by low dynamic range limitations. Some devices can handle up to a 10- or 12-bit dynamic range per channel. However, the human vision system can detect a wide luminance range of 14 orders of magnitude, which translates to around 46 bits. Luminance values in nature can be as high as 108 candela/m2 in bright sunlight, and as low as 10−6 candela/m2 on the underside of a rock on a moonless night.
High dynamic range (HDR) imaging presents a more versatile and natural image representation in line with the human vision system. HDR images can present a dynamic range higher than the traditional 8-bit, 10-bit and 12-bit representations to achieve a far higher image quality. HDR images can be used in the same kinds of devices and software tools that process conventional images if the HDR image format is compatible with the device or tool. Several HDR image formats have been developed, and cameras, computer graphics and display devices have begun to produce, process and display images with increasing dynamic ranges. HDR images also can be constructed from a collection of lower dynamic range images. See Debevec et al., “Recovering High Dynamic Range Radiance Maps from Photographs,” SIGGRAPH '97 (August 1997).
Various prior techniques have addressed problems associated with displaying or processing high resolution and/or high dynamic range images.
U.S. Pat. No. 7,120,293 describes an “interactive image” where each pixel location in the interactive image is assigned to one of several representative images having different characteristics. Image processing techniques are employed to determine which one of the several representative images represents the best exposure level or focus setting in connection with the selected pixel location.
U.S. Pat. No. 7,492,375 describes HDR viewers that allow one or more specific regions of interest in an HDR image to be selected and displayed. In the regions of interest, an HDR viewer displays the corresponding portion or portions of the HDR image. The corresponding portion or portions vary in some way from the original HDR image. For example, a portion of an HDR image corresponding to a region of interest can be tone-mapped to a different dynamic range.
A publicly available application called HDRView allows a user to open an HDR image on a lower dynamic range display and shift-click on a selected pixel location in an HDR image to vary the exposure of the image in accordance with the exposure level at the selected pixel location. A description of HDRView is available at http://athens.ict.usc.edu/FiatLux/hdrview/.
Engineers also use compression (also called coding or encoding) to reduce the amount of bits that are required to represent digital media such as HDR or SDR digital images. Compression decreases the cost of storing and transmitting a digital image by converting it into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original image from the compressed form. A “codec” is an encoder/decoder system.
Perceptible flaws in digital images after encoding or other processing are sometimes called artifacts, as they result from and show that the encoding or other processing occurred. These artifacts include blocking artifacts, banding artifacts and ringing artifacts.
Block patterns that are introduced by compression and appear in reconstructed images are often called block artifacts. Block artifacts can be especially noticeable in smoothly varying, gradient regions, such as an image of a clear sky. Block artifacts result, for example, from splitting an image into blocks for encoding, where the encoding includes a frequency transform process and quantization of AC coefficients for the blocks. Banding or contouring artifacts occur, for example, when sample values in an image are converted from a high bit resolution (e.g., 10 bits or 12 bits per sample value) to a lower bit resolution (e.g., 8 bits per sample value). When sample values are clipped to the lower bit resolution, steps between bands of values may become perceptible, especially in regions with smoothly changing sample values (e.g., a gradual transition from light to darker). Ringing artifacts can appear as a ripple pattern or other band of noise going away from the edge of the artifact into the background of the picture. Ringing artifacts can result from a frequency transform process and quantization for a block that includes the object or part of the object. Ringing artifacts can also be introduced at edges by excessive sharpening during editing.
Some post-processing approaches to controlling artifacts process images after decoding so as to smooth over or otherwise hide artifacts. Some systems adaptively filter across block boundaries to reduce the visibility of block artifacts. Other systems use dithering during post-processing to adjust the sample values of reconstructed pictures. For example, dithering can introduce small adjustments to values around a jagged edge so that the human viewer “averages” the values and perceives a smoother edge.
Whatever the benefits of previous techniques, they do not have the advantages of techniques and tools presented below.